On some interrelations of generalized qq-entropies and a generalized Fisher information, including a Cram\'er-Rao inequality

In this communication, we describe some interrelations between generalized $q$-entropies and a generalized version of Fisher information. In information theory, the de Bruijn identity links the Fisher information and the derivative of the entropy. We show that this identity can be extended to generalized versions of entropy and Fisher information. More precisely, a generalized Fisher information naturally pops up in the expression of the derivative of the Tsallis entropy. This generalized Fisher information also appears as a special case of a generalized Fisher information for estimation problems. Indeed, we derive here a new Cram\'er-Rao inequality for the estimation of a parameter, which involves a generalized form of Fisher information. This generalized Fisher information reduces to the standard Fisher information as a particular case. In the case of a translation parameter, the general Cram\'er-Rao inequality leads to an inequality for distributions which is saturated by generalized $q$-Gaussian distributions. These generalized $q$-Gaussians are important in several areas of physics and mathematics. They are known to maximize the $q$-entropies subject to a moment constraint. The Cram\'er-Rao inequality shows that the generalized $q$-Gaussians also minimize the generalized Fisher information among distributions with a fixed moment. Similarly, the generalized $q$-Gaussians also minimize the generalized Fisher information among distributions with a given $q$-entropy

An amended MaxEnt formulation for deriving Tsallis factors, and associated issues

An amended MaxEnt formulation for systems displaced from the conventional MaxEnt equilibrium is proposed. This formulation involves the minimization of the Kullback-Leibler divergence to a reference $Q$ (or maximization of Shannon $Q$-entropy), subject to a constraint that implicates a second reference distribution $P\_{1}$ and tunes the new equilibrium. In this setting, the equilibrium distribution is the generalized escort distribution associated to $P\_{1}$ and $Q$. The account of an additional constraint, an observable given by a statistical mean, leads to the maximization of R\'{e}nyi/Tsallis $Q$-entropy subject to that constraint. Two natural scenarii for this observation constraint are considered, and the classical and generalized constraint of nonextensive statistics are recovered. The solutions to the maximization of R\'{e}nyi $Q$-entropy subject to the two types of constraints are derived. These optimum distributions, that are Levy-like distributions, are self-referential. We then propose two `alternate' (but effectively computable) dual functions, whose maximizations enable to identify the optimum parameters. Finally, a duality between solutions and the underlying Legendre structure are presented.Comment: Presented at MaxEnt2006, Paris, France, july 10-13, 200

Quelques inégalités caractérisant les gaussiennes généralisées

International audienceDans cet article, on propose de caractériser les gaussiennes généralisées comme les densités atteignant les bornes de plusieurs extensions d'inégalités connues en théorie de l'information. On retrouve en cas particulier les résultats pour la gaussienne standard. Abstract - In this paper, we propose to characterize a class of generalized Gaussian distributions as the densities that saturate several extensions of classical information theoretic inequalities. The results for the standard Gaussian are recovered as a particular case

On escort distributions, q-gaussians and Fisher information

International audienceEscort distributions are a simple one parameter deformation of an original distribution p. In Tsallis extended thermostatistics, the escort‐averages, defined with respect to an escort distribution, have revealed useful in order to obtain analytical results and variational equations, with in particular the equilibrium distributions obtained as maxima of Rényi‐Tsallis entropy subject to constraints in the form of a q‐average. A central example is the q‐gaussian, which is a generalization of the standard gaussian distribution. In this contribution, we show that escort distributions emerge naturally as a maximum entropy trade‐off between the distribution p(x) and the uniform distribution. This setting may typically describe a phase transition between two states. But escort distributions also appear in the fields of multifractal analysis, quantization and coding with interesting consequences. For the problem of coding, we recall a source coding theorem by Campbell relating a generalized measure of length to the Rényi‐Tsallis entropy and exhibit the links with escort distributions together with pratical implications. That q‐gaussians arise from the maximization of Rényi‐Tsallis entropy subject to a q‐variance constraint is a known fact. We show here that the (squared) q‐gaussian also appear as a minimum of Fisher information subject to the same q‐variance constraint

Escort entropies and divergences and related canonical distribution

arXiv : 1109.3311International audienceWe discuss two families of two-parameter entropies and divergences, derived from the standard Rényi and Tsallis entropies and divergences. These divergences and entropies are found as divergences or entropies of escort distributions. Exploiting the nonnegativity of the divergences, we derive the expression of the canonical distribution associated to the new entropies and a observable given as an escort-mean value. We show that this canonical distribution extends, and smoothly connects, the results obtained in nonextensive thermodynamics for the standard and generalized mean value constraints

Entropies et critères entropiques

International audienceCe chapitre est centré sur les notions d'entropies et de lois à maximum d'entropie qui seront caractérisées selon plusieurs angles. Au delà des liens avec les applications en ingénierie puis en physique, on montrera qu'il est possible de bâtir des fonction-nelles régularisantes fondées sur l'emploi d'une technique à maximum d'entropie, qui peuvent alors éventuellement être utilisées comme potentiels ad hoc dans des pro-blèmes d'inversion de données. Le chapitre débute par un tour d'horizon des principales propriétés des mesures d'information, et par l'introduction de différentes notions et définitions. En particu-lier, on définit la divergence de Rényi, on présente la notion de distribution escorte, et on commente le principe du maximum d'entropie qui sera utilisé par la suite. On présente ensuite un problème classique d'ingénierie, le problème du codage de source, et on montre l'intérêt d'utiliser des mesures de longueur différentes de la mesure stan-dard, et en particulier une mesure exponentielle, qui conduit à un théorème de codage de source dont la borne minimale est une entropie de Rényi. On montre également que les codes optimaux peuvent être calculés aisément grâce aux distributions escortes. En section 1.4, on introduit et on étudie un modèle simple de transition d'état. Ce modèle conduit à une distribution d'équilibre définie comme une distribution escorte généra-lisée, et conduit en sous-produit, à nouveau à une entropie de Rényi. On étudie le flux d'information de Fisher le long de la courbe définie par la distribution escorte géné-ralisée, et on obtient des connections avec la divergence de Jeffreys. Finalement, on obtient différents arguments qui, dans ce cadre, conduisent à une méthode d'inférence fondée sur la minimisation de l'entropie de Rényi sous une contrainte de moyenne généralisée, i.e. prise vis-à-vis de la distribution escorte. À partir de la section 1.5.3, on s'intéresse alors à la minimisation de la divergence de Rényi sous une contraint

On generalized Cramér-Rao inequalities, generalized Fisher informations and characterizations of generalized q-Gaussian distributions

International audienceThis paper deals with Cramér-Rao inequalities in the context of nonextensive statistics and in estimation theory. It gives characterizations of generalized q-Gaussian distributions, and introduces generalized versions of Fisher information. The contributions of this paper are (i) the derivation of new extended Cramér-Rao inequalities for the estimation of a parameter, involving general q-moments of the estimation error, (ii) the derivation of Cramér-Rao inequalities saturated by generalized q-Gaussian distributions, (iii) the definition of generalized Fisher informations, (iv) the identification and interpretation of some prior results, and finally, (v) the suggestion of new estimation methods

À propos d’extensions de l’information de Fisher et de l’inégalité de Cramér-Rao

International audienceRésumé – On présente quelques extensions de résultats classiques de théorie de l'estimation, conduisant à définir de nouvelles formes de l'information de Fisher et à des généralisations de la borne de Cramér-Rao. On indique que ceci suggère une variation autour du maximum de vraisemblance. Ce cadre conduit également à une nouvelle caractérisation des gaussiennes généralisées, ainsi qu'à l'extension de certaines identités (de Bruijn) et inégalités (relations d'incertitudes). On retrouve en cas particulier les résultats habituels pour la gaussienne standard. Abstract – We present some extensions of classical results from estimation theory. This leads to new forms of Fisher information and t

Envelope and phase delays correction in an EER radio architecture

International audienceThis article deals with synchronization in the Envelope Elimination and Restoration (EER) type of transmitter architecture. To illustrate the performances of such solution, we choose to apply this architecture to a 64 carriers 16QAM modulated OFDM. We first introduce the problematic of the realisation of a highly linear transmitter.We then present the Envelope Elimination and Restoration solution and draw attention to its major weakness: a high sensitivity to desynchronization between the phase and envelope signal paths. To address this issue, we propose an adaptive synchronization algorithm relying on a feedback loop, a LeastMean Square formulation and involving an interpolation step. It enables the correction of delay mismatches and tracking of possible variations. We demonstrate that the quality of the interpolator has a direct impact on Error Vector Magnitude (EVM) value and output spectrum. Implementation details are provided along with an analysis of the behaviour and performances of the method. We present HPADS and Matlab simulation results and then focus on the enhancement of the transmitter performances using the proposed algorithm